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. r The^purpose of this st^ 

effectiveness of a tpken inqneta^ for improving resp^ 

rates to a mail questionnaire with a dff^^ 
pbstsecondary studfent population-. The results* demon 
u^ihg. a nibnetary inceittive , a substantial^ i^^ 

i-ate can be achieved at a reasp'nable cost per respondent without 
affecting the representativeness of the respondent grpupi In ^ 
aaaitibh, a single mailing which included an incentive was f pundit 
be much more effect iv^^^^^ elicitiiii responses than^ two mailings to an 
identical group which di<i^ n^ receive the incenti\|e, (Author) 
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University of Illihbis at Bhicagb 



The purpose. of this study was to evaluate the effectiveness, 
of ^ a token, monetary incentive , for ^improving response t'atSs^' 
to a mail questionnaire with a: difficult: , to survey 
pdstseeohda:ry student population. ^' ^ T^ results denibnstgate 
tha^ by using a monetary Iricentlve^ a^ subsfehtial 
imp^vemfent in^ response _2rateV^^^^ 6^ bf achieved ^ .^a 
r^sonable cost per- responden^t^^ without affecting the 
representativeness of the respdn^^ group. In'additipn,^a.. 
single mailing which included ah^ was fouHd tofbe " 

much^ more efi*ective_ at ' fieiiciting responses than two/ 
mailings to an identical group which did not i^eceive^ the 
incentive, ' : ' 





INTRODUCTION 



The need to attract an^ retalji'^s^ has become a dritical concern > 

for many, if hot mbst^^ educgftMrial institutibris. Furthermore^ 

: it. is clear that as^^^^^^^ pool ^^ traditional cM^ege age students diminishes, 
-^:;these issues will only increase in importance, ikistitutional researchers 
l^tudying recruitment and retention have found that student input is an ^ 
integral part of the e^ planriihg process. Decision-makers need 

information on students' backgrounds^ experiences, bpihiohs, and edueatibhal 
plans which, cannot be obtained solely from analyses of ihstitutibhal 
records. Therefore, research efforts using survey techniques play an 
important rdle^in providing data fbi? academic planning and policy • 
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development. 



Given the constraints of timi resources, and personnel, as \ielif^^ljt±iBi^^^ 
wide geographic dispersion pf students, at conifflufeer institutions, the mdi^y^)^^^^^ \ 



questionnaire may be the 
data, fl ma jpr drawback of 




ly acceptable -method of bbtaining student survey ^ff^ ""^^ 



the mail questionnaire, however > is p(^r response 

rate, which seriously affects the^ ability of the respondent group to > 

represent the characteristics and attitudes. of the survey population. It ;is 

. not Uflcdmmdn for institutional researchers to find their student survey 

i^^m)rts- hamper by response rates of 3G% to HG% (Matrbss, 1981 ), arid thi^ 

piSpblem may be compounded when the. subjects- lack ah association with_ the 

. • • ' ,* ■ ■ -. ■ (.^ ■ „ , ■ ■■ " 

.institution a4 the time /^f the s^tudy. Postsecondary students who have 

Jittle motivation to participatre^it^ research and have - 

: traditionally beeri^ difficult to survey ir>elud& those who faii ^fe^bpmplete 

/the application process, those who are admitted but do liot register for 

C^as^es, dropouts^ and alumni; It is theorized t^t 'potential and for 

students- are less likely to respond to questionnaires than current students^ 

because they have a limited interest in the subject matter (Heberleiti and 

Bauragartrier, 1978) ^ they lack Gommitm^ht to the goals arid success -of the ; 

institution, and they will hot be persohally affected by poif^ies and 

programs, developed as a result of survey findings (Linsky, 1975). 

■ ■ ' • ' ■ -, V. 

• I ■ . . • , ■ ' ... ■ ' 

' Numerolis techriiqUes for improving response rates can be found in the 
' research literature* Most are attempts to elicit responses by facilitating 
and/or motivating the completion ahd return of qpestiohhalres; Ainohg the * 
most successful are (1) mechanical and perceptual factors including 
pre-cbntacts^ iiiUltiple and intensive follow-ups^ and impressive d'r 
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/ from kh important official or organizatioji, "and , (3) direct motlvatidn^l 
factors in the fbriii of rewards (Linsky^ 1975) . i ^ 

■ ' Qrie of the most poiferful teehriiques for motivating survfy participation 

._ ^ _ . ' ' ■ j_ ' *_ 

. is the prepayment of token monetary incentives. Extensive reviews of the: 

literature on incentives were conducted by Armstrong (1975) and Linsky 

,(1975) who found that the Use of a monetary incentive resulted; in a /* 

' ' ■ . ' ■ . . . - 1^ 

ycbhsisteritly higher response rate when compared with a ebhtrbl group. In 
the tKeS^y-seven different cases examined by Armstrong (1975) and Llnsfcy 
; lA97S\r^ t increased by: an average of 20^5 through 'the use 

: bf ihceritjtV^s^ ahdj^^ sik instances the response rate for the group ^ 

■ ■''y^^'-,-^^ .." ----- ' - ■' . 

. receiving the than twice that bf the cbritrbl group. . ; 

Previous studf^s ba^^^^ incentives ranging from, one penny to one 

^©l^ar, but the-^u be the most cost-effective amount ^ 

.(Linsky^ 1975X. A greate!^^'.-^^ response rates has been 

demonstrated with bhe h^^ hbwever. (Armstrbng and Over tbn, 

1971;"Biumberg, eb al;^^^ ErdosV^y95lV Habkier Bourgette^ 1973; 

^ .. ^'-i- ^ --- ■- " ■ 

Mewman*;' 1962). Monetary incentives are most effective when prepaid rather 

t;|ian promised (Armstrong, 1975; Goodwin; 1979),' atnd when enclosed with the . 



first mailing bf the qUestibnnair'e father follow-Up (HUck arid 

Gleasbh, 1974). While ^here is ebhe-arh that the use bf ^tbkeri irieehtives 
introduces response bias, there is no evidence to supporfe^ this conclusion 
(Cox, 1976vNederhof, 1983; Wotruba, 1966). fe , ' :^ 

The few -feseaf chef s- whb bffef explanations fbf the i^UccessfUl Use of 
incen.trives tehjd to stress the symbalic rather than the monetary value of the /' 
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• the aGCbmplislM of an lmp^tarit task| rather than payment 'for the tiiiie 



and ihc^ohvehience/ involved ; in questionnaire 'compieti^ (tinsky; 1975 )*i .When 



an incentive, is- offered In advance, its acceptance creates a feeling of 
dbl:^atidn on the part v of the subject and- a need to reciprocate (Goddwinj 
19^9;^ Liriskyi 1975). A similar explanation holds that^l^^^^ 
distress, called pogriitive dissonance (Festinger', 1957) , is caused by the 
Lncbngruous actions of accepting the incentive i^ithout performing the 
requested service (Hackier and Bourgette, 1973)* This^ sense of cognitive 
dissonance can- be relieved by completing and returning the questionnaire. 
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Previous research on this particular method of response rate 
improvement may Ijack a theoretical foundation JCox, 1976; Heberlein and 
. Baumgartner^ 1978), but the evidence that ^monetary incentives have. a ' 

positive, arid often dramatic effect on response rates is bverwheiming.; Few 

■ ' _ ■ . :■■ ■ ■ ■ - • ■ ] :a\ 

studies have been published bh ihe^iitlves siriee_Jhe. middle 1970' s, rhbwever , 
. ■ ' ■ . - . ■ '. ■ ' . . ' 

and most were conducted in coniuriction with general public or commercial 

surveys. Only one study coUlp be located which investigated the use of 

incentives in a pbstsecbndary student survey, in thaE study of current 

students jwhb; were living on campus, Huek and Gleasbh (1974) f bund ^tha^ 

> fe^pbnse rate was increased by 1855 over a control group by , providing a 

, quarter incentive. ' 

There is a defiriite lack bf literature bn the Use bf incentives iri 

■ ■ ' • :v • • V ^' ■ - - - 

student surveys. Information on this topic would enable institutional . 
researchers to judge the merits of using this resport^e rate improvement 

technique within the higher education setting, and would be particularly 

' ■ ■ </ . ■"■ 

• ^ • . •' • ■ ' ' , 

■. • . ' ' ' ■ ■ ' '■ ■ . ■:• . • 




-ti3Miil-fQP--f'eseareher^s^^*h0-^%hep--^ 

■ :•. ■ ■ '•' ' 

from student^! whdvare unlikely td reMwndl.: • It was the, purpose df. this ,: . 
stjadyi aferefore, to provide' a ^ifiurr^enfe assessment of "th©j^effectiveness of a 
radnetaryvpiftC^ttftive with.a dimau|t;td sur^f% postsecohdai*y student/-/ 
populat'ioriv-and to determine if/ap improvement in response rate couid fe" 
jdbtained: at fa" reasonable cost ',per' y^espondent without affecting 'the 
represientativeness^^f th/ respondent ^ * - ' 

I ■ METHODOLOGY . 

/ . / ^ * ' ' : • : ■ . _ _ : - 

■ ThiS steady, was 'eondiieted /Wtf /a large , public^ f our-yeir postsecondary 

/ ./ ' • - :_ . • > 

ed^uc^tionil institution."^ ftli 371 undergraduate transfer students who first 



^nro±ied for the Fall 1982 term and voluntarily withdrew before the 

beginning of the next term were included as subjects. ; ^ 

■ . ...t. / • ■ ■ ■ ' ■ . • . ' 

A simple t'andom sanjfDle, stratified ybri academic statu 

probation) wis utilized to select 200 Jexperlm^htaI subject^^^^ members 

of a control p?oUp.* Each subject repeived a twb-^^^ questionnaire ^ . 

cohtairiirigbriiy research quest . 

demographi^^'a'eademie^^ chai?*acteristics from institutional 

records, students were asked to identify themselves by name and social 

security number. Ninety- three percent of the students complied with this- 

request^ /and there was ho .difference between the experimental and cQhtrol 

:'oups in this regard. Because data were cdlle from these twS sources, 



' _7 ■ ._ -' — _ ■ 1- _ ' V-l ■ ■•■■.■*-■--_--_- — - ^ -■ 

*The sum of . $200 in non-state funds was made available for= this project 
^rcHn the university foundation. This amount enabled the researchers to 
dff'er the one dollar incentive to each of 200 subjects, and thus accounts 
fcif the slight disi^epancy in the size of experimental and cdfitrol,* groups.. . 



-aet^f'a t^iitfbfHnaM ^ri was availabl e on th e characberistics of both 



respondents arid noriresponderits ^ the quest ibririaire was shortened 
considerably, and the overall survey costs were reduced by mailirig the 
follow-up only to nonrespondentis • A. one-word instruction at the bottom of 
the first page of* the questionnaire,^ presented i^ formats, was used 

feo/distiriguish between both the experimental arid control grg)Aips and the 
respondents to the -first irid secopd mailirigs. Go ver letters had 
cfiersonalized inside addresses arid salutatioris, arid were sigried by a key 
ojpgjLci^l of the Qffice of Academic Affairs. The letters emphasized the ; 
importariee of thre survey prbject arid the opportunity for former students to ; 
express their opiriioris to the uriiversity admiriistratibri. Confidentiality of 
response was guaranteed. Each member of the ^experimerifcal group reeeiyed brie 
ddliar in cash as an incentive^ A postscript to their cover letters stated . 
that the dollar was iriterided as a small token of appreciation for .their 
assisfcgtffce. .finvSlopes 'w6re harid-stamped arid addresses were typec|. / 
Self-addressed, postage-paid *returri envelopes' provided. - 

• A folldw-up was conducted six weeks after the original mailirig ^ Each 
student who had riot respo^ by that time of who had responded anonymously , 
was sent a fblldw-iap packet which ebrif^iried^a letter explaining the 
importance of their response to the fesearch pro^eet^ bf ' the 

cjuestionnaire, and a return envelope. Members of expeririientai grotap did::rib|; • 
receive aribther 'iriceritiVe .. Aithpuph a variety of methods whlqji are ^ 

^_ ■_ ' - • ■■ ■ : ^'a ; _ •_ - ■ ; _« ' '.- - - - ' - - - -■ ^- 

' considered effective for imprbvirig resporise rates were Utilized, including 
personalization of survey materials arid multiple cbritaets, studerits iri both 
the experimental and control- groups ^ were Ir^eated ideritidaily with the 



FINDINGS 



The f iridirigs, of this study cbricerri the effect-bf* the incentive 
response rate, ^represehtativeh^Sf^ ^ survey eost^. The bverall survey 
response rate was 54.7^^ with 196 students respbhdihg but of 358 who 
received questionnaires. (Qu^ for the thirteen additional 

students eould riot be delivered^ eight from the experimental group and five 
from the^^ritroi group.) The first mail i\ig prpduc^^ 

39. l^S, and 15.6^ responded to the foiiow-jp^ As shown^ in Table 1 , there 

were large differences between the experimental arid cori'troi groups in tdtal 

■ • " • ■ r ' ' ^ . 

response rate arid response to the first mailing and the f"Qllow-up. The 
total response rate for the experimerital group was 63.5^* and riearly all, of 
these subjects resporided to the first mailing. A respbrise rate of 54.2^ was 
achieved with one mailing to this group, arid an additibrial 9^4% responded tb 



table 1. Survey Respbrise Rates 



-Survey Group 



Hailing 



Experimental 



ebritfbl 



First Mailing 
Folldff-up feiling 



54. 2^ 
9.451 



,21.7^ 
22 .956 



TOTAL 



63.55t 



44.651 



the follow-up. The total response rate for tfte cbhbrbi group was 44;69>> 



Approximately half of thesetstudents-respipnded to the first mailing (21.7%) 
ami half to the follow-up (22.9?5). 




In braer to Judge the*re^reserifcativehess or fe^^ survey^re^pbridehts, 

. . ' • ' _'_ [ ■ '_t_ _ _ _i _ _ -_ _ 1 ^ -'-^ - ______ - 

members of the experimental and control 'poptfiatiohs wene compared with 

respondents from the Corresponding groups on fifteen deiflographic, academic, 

performance^ and high schddi background chara^ SPSS / / 

(statistical Package fbr"^ the Social Sciences )> t- tests were perfbrmed^ fe^^^ ; 

interval levjsl variaBies, and the response categories of each nQh-inte^ySi^^^^^^^ 

level variable were converted into dymmy : variables for: cy.^erenbe 

■ -^_ r'. _ ^ . • ,^ . ' '------I- 

proportions tests (Ldetfeerj-a^^ 198p). No statisti^caily 

Significant differences < ,05)^were found between the pogiittt^idri and v 
r^spbhdehts of either the experimental group or the con,fef*bl group bh; any of < 
the fifteein <dharacteris^ic§. The respondents ^frbm bo t^^ . '^i 

ther^efbr^ cdnsider^d ^ of their respective ipopi^atf^^ 



■9 -.' , 

-/■;_■'■•' 



Rfespbftdents to the first mailing frdm\bbth jthe experi^^ 



■I- -V: 



groups were also cbmparedy^w^ their reaped ti^e pibpuialid^ to - 

determine representativeness i Males were uhderrepre3ehted aa first mailing 
respondent3,-from the ^peMmental group (p = .03951^^^ 



uriderrepreserited as f^jcst^fiallis^ from the cqntfol group (p 7= .028)1 



There were ho other >^tistieally|signif leant dif fere between; the 
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*These character is tics included five demograph|c variable 
status^ age^ residence^ and cit5;2enship)^^^. acad^ib:: var^i^l^^ 
college, credit hdurs^ and day dr evening 7s taly^ 

academic perfbrmahde (academtc statu^ i grad^ pidint 7|ii^erage ACT "^«cdreli ( 
High school backgrbuhd ineludeS^ ,thdv type^ ai:^^ ^li^h 
school -attended i. ; ;'r':7'. 'V-?'-^^^':'^-.- ' i ''-'^f^.- 



1 \ :// 



V7i ; 



^^"pdpjL^^ hespondents tb the first iftailing. Exper/imental and contrbF 

grqup|Vesporiderits wer^cbmpared on questionnaires resp^^ to 
demographic, aeademie^ arid performanee eharaetefi^fciesi 0^ 
• s^ emerged betweeri the two re3^^ .011); 

■ with a much larger* percentage of experimental-respondents stating that they 
had withdrawn from the universi£y after completing all desired courses* 
^ This was only one of seventeerh questions on. the survey instrument, and any 

-''^l' _ •' ^ ^_ •_ _ _• ^- __ •' _\ " _ _ • " . - 

conclusion tft'at response bias was'ihtroduded by offering; the irieeritive 

v ••• . ■ • ./,,,. ■ . . • 

- , . .■ , : _• . .V...- . r .■:/■■•/*■■'.■-.■ --.v ■__ j /v' -i ■ 

cannot be based on this finding alprie;. In Xact, from ft r^earch perspective 
It is an advantage that the ince/tivi jotl 

'particular students to respond/ 'Tiiere were lio statistically significant 
?diffipences between m^ and control groups who 

J responded tothe first^mailing^ : 

i i- . survey Vests JSr s^ and postage totaled 63 cents per 

• ; subject for the ephtrol -group and, with the inclusion of the one dollar 
/ I iriGehtive; $1.63 -pe? subject for: the experimeriEai groups Each returned 

; ' guesti(pni^a|re added^ 24^^^ costs. Survey costs per • 

> respondent were' caMliiated by 'dividing the total costs for mailing and 
■ turning the vqtjesUonnaire by the number of respondents in that; group. 
T rTdfeai costs were $3^45 peP^^'ei^bhdSnt:. for the experimeijS^l group and 'IS. 85. 
' per i respondent fbr^^^t^^ This' idi^f^^erence regfeisents an overall ^ 

cost per v§xp^Mmental^ r^ than the cost per control^^^^^ : ^ 

: grbjtip respohHeh;t:.^^^ T^^ cost of ihe first mailing to the .experimental •gr^dtip 

■*■•■ . ^ • . ' > ■ -"'V' ' ■ 

was $3i3t ger respondent. .Even though this figtare includes the cost of bhe^^^^^^^^^^ 

• //incentive^ it was only' 4^ Higher t^^ the ~f¥rit mailing cost of |3«23 'f^^ 



, .._ . , . . . ^ . .... ...... . . .... ■:r::^^ 
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respdh^efit ^f^^ the control gpoup. Costs for the fbllbw-up maiiing were ,mucH_ 

—. ■ ^ • •' a . - ' \ ■ ■ ■ ^ ^ ■ ' ' • . . . ■ . ■•. . 

higfier [for the experimental group (.$3-91 ) than for the eontrol groUp . 

i'^^ respondent -was due primarily^ to : 



■■1 ■•■ 



the very sMll percents^ of .subjects Rho responded to the follow-^up,: but> \ 
alsarreflects the cost of sertdirig the fbllow-up^packet to a numbei? of 
subjeets who shortly thereafter, returned^ questionnaires from the first ; . / 
mailing and Ehus ; were not qoun ted aS -^^^ respondents i ^ eomiiarisoh^ of 

the survey costs per respondent for both the experimental and control-grdups 

- ' l • --'-^ - \i' ' V' ■ - ■ — ' - : ' • ■ ^ _ ■■ ■ . . ■ ■ . ~. - ■ **• - -. 

arid .the Cwor mailings 'are ;^bwh in Table . 2. Although personnel costs wer.e ' , • 
hot ealeulated^ these "costs '"would f?ave been. -higher for the control group due 
to thte necessity of. niaiiing the foi3^w.T^ to a much larger number of \ ^ 

^ ' V* Tp ;su^mg!ri^^^^^^^^ defe^rmin^d* that trie Use 

of a token monetary incentive was an feffeetive methoU for mptiyatihg 



Table* 2. 



Costs per Respbn^^^t' 











Survey Group 


Mailing ^ i - v^. 


; V Experimental Control 







First,Miillng ■ , 
Fofidw^up JHaiiiri^ 



$3.37* 
• $3.91 ; 



$3.23 
12.49 



TOTAL 



$3.45* 



$2>85' 



■ ; ; ; - ;*inciU(ie^ . . cosi of .; $t' ihcentive . f or each idf the; ?Qp sub jecfcs 



11 



responses lib a , mail questionnaire frdiri sfe'udents who H##r#raditibnally ^een ' 



difficuib to survey. Hie response rata of bJTSSt which tas achieved by using 
an Incentive; was iidt only a subsbantaal improvement over^^thab^^S^ a conbrol 
group/ bub: -wglsaiiso as large as response- raters of ben f oufi#'& Qampus 

survey^ iMatrti^s^^ I98I ) v This Is cibnside^ good response riabe. fdt a maii 
questiorihalre (Lin, 1975; Warwick and Lihirigerv 1975) , albhdugH using fcl 
inGefibi\^,e along^ other bechhiquls such as aadibio^ by mail' 

or betephorie may have resulb^^ -I 
quesbiprihaii^^^^ of bhe respdndenb group was hob y 

af f ebbed j and the behefifc of using bhe ;incenbiy« far outweighed bhe slightly ^ 
'higher.- cosb;' ':" / ■ . ■■/■'■/_'.]'/[■. • 



y T)^ sbriking When the experimental 

and corttrbi; groups are cdmpare^^ on the first mailing alone. As Table 3 
shows, o^er half of the subjects who received the ihbehtlve (54.21), but 
less than:;a quarter of the control group (21.7^5), responded to the first 



Table 37 A Gomparisphvd^ and Survey Costs, 



V Gom^ Factor 



Survey Group 



Experimental ^ 



Gonferolv V o ^ ^ 




Response Ra te 

Gosb per Respondent 



$3-37* 



EKLC 



*lnclude^ cost o'f $1 inc^htive for tkoh< of thS 200 iubjSctsV 



IS 



mailing. By using an incentive the r^^^^ was improved by 32.5%, ^ a^^^ 

a^hdugh the response rate of the ^xp^^ group was one and a half 

times larger than that of the control group, it Sa^ achieved at a nearly . 
identical cost per respbhderit. Furthermore^ the time^ effort, ana expense ;•. 
of sending a foiiow-up ^ the ebhtrol group, while greatly increasing the 
final response rate, did not prove to Be Seari^ as effective ait elieitirig 
respbrises as one mailing wh^ an incentive. This finding provided 

evidertee that the gains iri response rates which result from using a monfetary 
incentive may enable researchers to dispense with follbw-Up 
; altogether, thus reducing survey-reiated personnel costs considerably i 

.CONGLUSfON^^^^ ? ^ 

Institutional planners have the rar^ bppbi'tuhity tb cbriduct researbh bn^ 
populations which can be enumerated- individual by individuai, arid for^ which 
a wide variety of accurate information is available at the ouEsejt. . this 
ability tb identify the research subjects, as weir as level 
of the subjects i the geographic dispersibri bf student and ftjrmer student ; 
populations, and the high cost of. personal interviews, niake 
questionnaire a natural choice for Vthe collection of yvita^ 
Attempts tb utilize the mail questionnaire with student populations are 
often frustratirigly unsuccessful^ however ^ bepaUse of the difficulfcjr in 
motivating survey participation arid the low response rates which bccui? as a 
result. ^ This problem miiy be exacerbated wheri the resea rid . 

longer assbciated with thp . institution* : ^ . ' : 

the need for high quality data from students who have traditibnally 



l3eeri difrieult ^ prompted this exploration of the effectivehess of 

token monetai*y incehtives for improving mail questionnaire response ratesi 
In this study of former studenEs, ■ the use of^ 

m a response rate improvement of 9^ over a ccmtfol gfeoup- without . . 

affecting the represeritativ^nBss^ of the respondents or biasing responses. ; 
• The additional cost of providing the incentive was negligible when Xiomparecl 

with the increase^^ response rate • These findings are even more' 

significant when only the^first mailing is considered, ' The evidence e|asts 
. in this study that the use of ^ token monetary incentive may eliminate 

aitogether the heed for follow-ups, which are particularly costly from the 
* vistandpoint of time and personnel. 

. these results confirm the findings of p^/ious studiesy most of which 
were published over a decade ago, that .prepaid incentives are useful tools 
for improving- the response rates of mall questionnaires. In additibh^ the 
monetary incentive was 3hown to be as effective at motivating ^^^r 

, a postsecdndiry student, survey population as was demonstrated in the past 
with the general public arid various commercial and occupational populations. 
The tokieri monetary irieeriiive is riot drily effective^ but by •elimiriatirig 
unsuccessful researcfi efforts which result from poor response rates arid by 
reducing the survey-related costs associated wit^ it is 

, also a cbstreffec researchers who depend on 

student input to form the guideliries for decision-making on academic 
planning arid policy issues will find the monetary ineeritive to be a 

: -worthwhile technique tot enhancing the quality of survey . data collected ■ 
through mail questionnaires. 
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